
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>Running multiple parsers on an input file &#8212; Universal Ctags 0.3.0 documentation</title>
    <link rel="stylesheet" href="_static/classic.css" type="text/css" />
    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    './',
        VERSION:     '0.3.0',
        COLLAPSE_INDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  true,
        SOURCELINK_SUFFIX: '.txt'
      };
    </script>
    <script type="text/javascript" src="_static/jquery.js"></script>
    <script type="text/javascript" src="_static/underscore.js"></script>
    <script type="text/javascript" src="_static/doctools.js"></script>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="next" title="Building ctags" href="building.html" />
    <link rel="prev" title="Choosing a proper parser in ctags" href="guessing.html" /> 
  </head>
  <body>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="building.html" title="Building ctags"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="guessing.html" title="Choosing a proper parser in ctags"
             accesskey="P">previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="index.html">Universal Ctags 0.3.0 documentation</a> &#187;</li> 
      </ul>
    </div>  

    <div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body" role="main">
            
  <div class="section" id="running-multiple-parsers-on-an-input-file">
<h1>Running multiple parsers on an input file<a class="headerlink" href="#running-multiple-parsers-on-an-input-file" title="Permalink to this headline">¶</a></h1>
<p>Universal-ctags provides parser developers ways(guest/host and
sub/base) to run multiple parsers for an input file.</p>
<p>This section shows concepts behind the running multiple parsers,
real examples, and APIs.</p>
<div class="section" id="applying-a-parser-to-specified-areas-of-input-file-guest-host">
<span id="host-guest-parsers"></span><h2>Applying a parser to specified areas of input file (guest/host)<a class="headerlink" href="#applying-a-parser-to-specified-areas-of-input-file-guest-host" title="Permalink to this headline">¶</a></h2>
<p>guest/host combination considers the case that an input file has areas
written in languages different from the language for the input file.</p>
<p><em>host parser</em> parses the input file and detects the areas.
<em>host parser</em> schedules <em>guest parsers</em> parsing the areas.
<em>guest parsers</em> parses the areas.</p>
<p>guest parsers are run only when <cite>–extras=+g</cite> is given.  If
<cite>–fields=+E</cite> is given, all tags generated by a guest parser is marked
<cite>guest</cite> in their <cite>extras:</cite> fields.</p>
<div class="figure">
<a class="reference internal image-reference" href="_images/area-and-parsers.svg"><img alt="_images/area-and-parsers.svg" src="_images/area-and-parsers.svg" /></a>
</div>
<div class="section" id="examples-of-guest-host-combinations">
<h3>Examples of guest/host combinations<a class="headerlink" href="#examples-of-guest-host-combinations" title="Permalink to this headline">¶</a></h3>
<div class="section" id="css-javascript-html-parser-combination">
<h4>{CSS,JavaScript}/HTML parser combination<a class="headerlink" href="#css-javascript-html-parser-combination" title="Permalink to this headline">¶</a></h4>
<p>For an html file, you may want to run HTML parser, of course. The
html file may have CSS areas and JavaScript areas. In other hand
Universal-ctags has both CSS and JavaScript parsers. Don’t you
think it is useful if you can apply these parsers to the areas?</p>
<p>In this case, HTML has responsible to detect the CSS and
JavaScript areas and record the positions of the areas.
The HTML parser schedules delayed invocations of CSS and
JavaScript parsers on the area with promise API.</p>
<p>Here HTML parser is a host parser. CSS and JavaScript parsers
are guest parsers.</p>
<p>See <a class="reference internal" href="parser-html.html#html"><span class="std std-ref">The new HTML parser</span></a> and parsers/html.c.</p>
</div>
<div class="section" id="c-yacc-parser-combination">
<h4>C/Yacc parser combination<a class="headerlink" href="#c-yacc-parser-combination" title="Permalink to this headline">¶</a></h4>
<p>A yacc file has some areas written in C. Universal-ctags has both YACC
and C parsers. You may want to run C parser for the areas from YACC
parser.</p>
<p>Here YACC parser is a host parser. C parser is a guest parser.
See <a class="reference internal" href="internal.html#promiseapi"><span class="std std-ref">promise API</span></a> and parsers/yacc.c.</p>
</div>
<div class="section" id="pod-perl-parser-combination">
<h4>Pod/Perl parser combination<a class="headerlink" href="#pod-perl-parser-combination" title="Permalink to this headline">¶</a></h4>
<p>Pod (Plain Old Documentation) is a language for documentation.  The language
can be used not only in a stand alone file but also it can be
used inside a Perl script.</p>
<p>Universal-ctags has both parsers for Perl and Pod.
The Perl parser recognizes the area where Pod document is
embedded in a Perl script and schedules applying pod parser
as a guest parser on the area.</p>
</div>
</div>
<div class="section" id="api-for-running-a-parser-in-an-area">
<h3>API for running a parser in an area<a class="headerlink" href="#api-for-running-a-parser-in-an-area" title="Permalink to this headline">¶</a></h3>
<p><a class="reference internal" href="internal.html#promiseapi"><span class="std std-ref">promise API</span></a> can be used.
A host parser using the interface has responsibility to detect areas
from input stream and record them with name of guest parsers that will
be applied to the areas.</p>
</div>
</div>
<div class="section" id="tagging-definitions-of-higher-upper-level-language-sub-base">
<span id="base-sub-parsers"></span><h2>Tagging definitions of higher(upper) level language (sub/base)<a class="headerlink" href="#tagging-definitions-of-higher-upper-level-language-sub-base" title="Permalink to this headline">¶</a></h2>
<div class="section" id="background">
<h3>Background<a class="headerlink" href="#background" title="Permalink to this headline">¶</a></h3>
<p>Consider an application written in language X.  The application has
its domain own concepts. Developers of the application may try to
express the concepts in the syntax of language X.</p>
<p>In language X level, the developer can define functions, variables, types, and
so on. Further more, if the syntax of X allows, the developers want to
define higher level(= application level) things for implementing the
domain own concepts.</p>
<p>Let me show the part of source code of SPY-WARS, an imaginary game application.
It is written in scheme language, a dialect of lisp.
(Here <a class="reference external" href="http://practical-scheme.net/gauche/index.html">gauche</a> is considered
as the implementation of scheme interpreter).</p>
<div class="highlight-scheme"><div class="highlight"><pre><span></span><span class="p">(</span><span class="k">define </span><span class="nv">agent-tables</span> <span class="p">(</span><span class="nf">make-hash-table</span><span class="p">))</span>
<span class="p">(</span><span class="nf">define-class</span> <span class="nv">&lt;agent&gt;</span> <span class="p">()</span>
  <span class="p">((</span><span class="nf">rights</span> <span class="nv">:init-keyword</span> <span class="nv">:rights</span><span class="p">)</span>
   <span class="p">(</span><span class="nf">responsibilities</span> <span class="nv">:init-keyword</span> <span class="nv">:responsibilities</span><span class="p">)))</span>

<span class="p">(</span><span class="nf">define-macro</span> <span class="p">(</span><span class="nf">define-agent</span> <span class="nv">name</span> <span class="nv">rights</span> <span class="nv">responsibilities</span><span class="p">)</span>
  <span class="o">`</span><span class="p">(</span><span class="nf">hash-table-put!</span> <span class="nv">agent-tables</span> <span class="ss">&#39;,name</span>
                    <span class="p">(</span><span class="nf">make</span> <span class="nv">&lt;agent&gt;</span>
                      <span class="nv">:rights</span> <span class="ss">&#39;,rights</span>
                      <span class="nv">:responsibilities</span> <span class="ss">&#39;,responsibilities</span><span class="p">)))</span>

<span class="p">(</span><span class="nf">define-agent</span> <span class="nv">Bond</span> <span class="p">(</span><span class="nf">kill</span> <span class="o">...</span><span class="p">)</span> <span class="o">...</span><span class="p">)</span>
<span class="p">(</span><span class="nf">define-agent</span> <span class="nv">Bourne</span> <span class="o">...</span><span class="p">)</span>

<span class="o">...</span>
</pre></div>
</div>
<p><cite>define</cite>, <cite>define-class</cite>, and <cite>define-macro</cite> are keywords of scheme
for defining a variable, class and macro. Therefore scheme parser of
ctags should make tags for <cite>agent-tables</cite> with variable kind,
<cite>&lt;agent&gt;</cite> with class kind, and <cite>define-agent</cite> with macro kind.
There is no discussion here.</p>
<blockquote>
<div>NOTE: To be exactly <cite>define-class</cite> and <cite>define-macro</cite> are not the part
of scheme language. They are part of gauche. That means three parsers
are stacked: scheme, gosh, and SPY-WARS.</div></blockquote>
<p>The interesting things here are <cite>Bond</cite> and <cite>Bourne</cite>.</p>
<div class="highlight-scheme"><div class="highlight"><pre><span></span><span class="p">(</span><span class="nf">define-agent</span> <span class="nv">Bond</span> <span class="p">(</span><span class="nf">kill</span> <span class="o">...</span><span class="p">)</span> <span class="o">...</span><span class="p">)</span>
<span class="p">(</span><span class="nf">define-agent</span> <span class="nv">Bourne</span> <span class="o">...</span><span class="p">)</span>
</pre></div>
</div>
<p>In scheme parser level, the two expressions define nothing; the two
expressions are just macro(<cite>define-agent</cite>) expansions.</p>
<p>However, in the application level, they define agents as the
macro name shown. In this level Universal-ctags should capture
<cite>Bond</cite> and <cite>Bourne</cite>. The question is which parser should
capture them?  scheme parser should not; define-agent is not part of
scheme language. Newly defined SPY-WARS parser is the answer.</p>
<p>Though <cite>define-agent</cite> is just a macro in scheme parser level,
it is keyword in SPY-WARS parser. SPY-WARS parser makes a
tag for a token next to <cite>define-agent</cite>.</p>
<p>The above example illustrates levels of language in an input
file. scheme is used as the base language. With the base language we
can assume an imaginary higher level language named SPY-WARS is used
to write the application. To parse the source code of the application
written in two stacked language, ctags uses the two stacked parsers.</p>
<p>Making higher level language is very popular technique in the
languages of lisp family (see <a class="reference external" href="http://www.paulgraham.com/onlisp.html">On Lisp</a> for more details).
However, it is not special to lisp.</p>
<p>Following code is taken from linux kernel written in C:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="n">DEFINE_EVENT</span><span class="p">(</span><span class="n">mac80211_msg_event</span><span class="p">,</span> <span class="n">mac80211_info</span><span class="p">,</span>
        <span class="n">TP_PROTO</span><span class="p">(</span><span class="k">struct</span> <span class="n">va_format</span> <span class="o">*</span><span class="n">vaf</span><span class="p">),</span>
        <span class="n">TP_ARGS</span><span class="p">(</span><span class="n">vaf</span><span class="p">)</span>
<span class="p">);</span>
</pre></div>
</div>
<p>There is no concept EVENT in C language, however it make sense in the
source tree of linux kernel. So we can consider linux parser, based on
C parser, which tags <cite>mac80211_msg_event</cite> as <cite>event</cite> kind.</p>
</div>
<div class="section" id="terms">
<h3>Terms<a class="headerlink" href="#terms" title="Permalink to this headline">¶</a></h3>
<div class="section" id="base-parser-and-subparser">
<h4>Base parser and subparser<a class="headerlink" href="#base-parser-and-subparser" title="Permalink to this headline">¶</a></h4>
<p>In the context of the SPY-WARS example, scheme parser is called a <em>base
parser</em>. The SPY-WARS is called a <em>subparser</em>. A base parser tags
definitions found in lower level view. A subparser on the base parser tags
definitions found in higher level view. This relationship can be nested.
A subparser can be a base parser for another sub parsers.</p>
<div class="figure">
<a class="reference internal image-reference" href="_images/stack-and-parsers.svg"><img alt="_images/stack-and-parsers.svg" src="_images/stack-and-parsers.svg" /></a>
</div>
<p>At a glance the relationship between two parsers are similar to the
relationship guest parser and host parser description in
<a class="reference internal" href="#host-guest-parsers"><span class="std std-ref">Applying a parser to specified areas of input file</span></a>.  However, they are different. Though a guest
parser can run stand-alone, a subparser cannot; a subparser needs help
from base parser to work.</p>
</div>
<div class="section" id="top-down-parser-choice-and-bottom-up-parser-choice">
<h4>Top down parser choice and bottom up parser choice<a class="headerlink" href="#top-down-parser-choice-and-bottom-up-parser-choice" title="Permalink to this headline">¶</a></h4>
<p>There are two ways to run a subparser: top down or bottom up parser
choices.</p>
<p>Universal-ctags can chose a subparser <a class="reference external" href="guessing">automatically</a>.
Matching file name patterns and extensions are the typical ways for
choosing. A user can choose a subparser with <cite>–language-force=</cite> option.
Choosing a parser in these deterministic way is called <em>top down</em>.
When a parser is chosen as a subparser in the top down way, the
subparser must call its base parser. The base parser may call methods
defined in the sub parser.</p>
<p>Universal-ctags uses <em>bottom up</em> choice when the top down way
doesn’t work; a given file name doesn’t match any patterns and
extensions of subparsers and the user doesn’t specify
<cite>–language-force=</cite> explicitly. In choosing a subparser bottom up way
it is assumed that a base parser for the subparser can be chosen
by top down way. During a base parser running, the base parser tries
to detect use of higher level languages in the input file. As shown
later in this section, the base parser utilizes methods defined in its
subparsers for the detection. If the base parser detects the use of a
higher level language, a subparser for the higher level language is
chosen.  Choosing a parser in this non-deterministic way(dynamic way)
is called <em>bottom up</em>.</p>
<p>Here is an example. Universal-ctags has both m4 parser and Autoconf
parser.  The m4 parser is a base parser. The Autoconf parser is a
subparser based on the m4 parser. If <em>configure.ac</em> is given as an
input file, Autoconf parser is chosen automatically because the
Autoconf parser has <em>configure.ac</em> in its patterns list. Based on the
pattern matching, Universal-ctags chooses the Autoconf parser
automatically(top down choice).</p>
<p>If <em>input.m4</em> is given as an input file, the Autoconf parser is
not chosen. Instead the m4 parser is chosen automatically because
the m4 parser has <em>.m4</em> in its extension list. The m4 parser passes
every token finding in the input file to the
Autoconf parser. The Autoconf parser gets the chance to probe
whether the Autoconf parser itself can handle the input or not; if
a token name is started with <cite>AC_</cite>, the Autoconf parser
reports “this is Autoconf input though its file extension
is <cite>m4</cite>” to the m4 parser. As the result the Autoconf parser is
chosen(bottom up choice).</p>
<p>Some subparsers can be chosen both top down and bottom up ways. Some
subparser can be chosen only top down way or bottom up ways.</p>
</div>
<div class="section" id="exclusive-subparser-and-coexisting-subparser">
<h4>Exclusive subparser and coexisting subparser<a class="headerlink" href="#exclusive-subparser-and-coexisting-subparser" title="Permalink to this headline">¶</a></h4>
<p>TBW. This must be filled when I implement python-celery parser.</p>
</div>
</div>
<div class="section" id="api-for-making-a-combination-of-base-parser-and-subparsers">
<h3>API for making a combination of base parser and subparsers<a class="headerlink" href="#api-for-making-a-combination-of-base-parser-and-subparsers" title="Permalink to this headline">¶</a></h3>
<div class="section" id="outline">
<h4>Outline<a class="headerlink" href="#outline" title="Permalink to this headline">¶</a></h4>
<p>You have to work on the both sides: a base parser and subparsers.</p>
<p>A base parser must define a data structure type(<cite>baseMethodTable</cite>) for
its subparsers by extending <cite>struct subparser</cite> defined in
<em>main/subparser.h</em>.  A subparser defines a variable(<cite>subparser var</cite>)
having type <cite>baseMethodTable</cite> by filling its fields and registers
<cite>subparser var</cite> to the base parser using dependency API.</p>
<p>The base parser calls functions pointed by <cite>baseMethodTable</cite> of
subparsers during parsing. A function for probing a higher level
language may be included in <cite>baseMethodTable</cite>.  What kind of fields
should be included in <cite>baseMethodTable</cite> is up to the design of a base
parser and the requirements of its subparsers. A method for
probing is one of them.</p>
<p>Registering a <cite>subparser var</cite> to a base parser is enough for the
bottom up choice. For handling the top down choice (e.g. specifying
<cite>–language-force=subparser</cite> in a command line), more code is needed.</p>
<p>call <cite>scheduleRunningBaseparser</cite> function from a function(<cite>parser</cite> method)
assigned to <cite>parser</cite> member in <cite>parserDefinition</cite> of the subparser,
<cite>scheduleRunningBaseparser`is declared in *main/subparser.h*.
`scheduleRunningBaseparser</cite> takes an integer argument
that specifies the dependency used for registering the <cite>subparser var</cite>.</p>
<p>By extending <cite>struct subparser</cite> you can define a type for
your subparser. Then make a variable for the type and
declare a dependency on the base parser.</p>
</div>
<div class="section" id="details">
<h4>Details<a class="headerlink" href="#details" title="Permalink to this headline">¶</a></h4>
<div class="section" id="fields-of-subparser-type">
<h5>Fields of <cite>subparser</cite> type<a class="headerlink" href="#fields-of-subparser-type" title="Permalink to this headline">¶</a></h5>
<p>Here the source code of Autoconf/m4 parsers is referred as an example.</p>
<p><em>main/types.h</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">struct</span> <span class="n">sSubparser</span><span class="p">;</span>
<span class="k">typedef</span> <span class="k">struct</span> <span class="n">sSubparser</span> <span class="n">subparser</span><span class="p">;</span>
</pre></div>
</div>
<p><em>main/subparser.h</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">typedef</span> <span class="k">enum</span> <span class="n">eSubparserRunDirection</span> <span class="p">{</span>
        <span class="n">SUBPARSER_BASE_RUNS_SUB</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">&lt;&lt;</span> <span class="mi">0</span><span class="p">,</span>
        <span class="n">SUBPARSER_SUB_RUNS_BASE</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">&lt;&lt;</span> <span class="mi">1</span><span class="p">,</span>
        <span class="n">SUBPARSER_BI_DIRECTION</span>  <span class="o">=</span> <span class="n">SUBPARSER_BASE_RUNS_SUB</span><span class="o">|</span><span class="n">SUBPARSER_SUB_RUNS_BASE</span><span class="p">,</span>
<span class="p">}</span> <span class="n">subparserRunDirection</span><span class="p">;</span>

<span class="k">struct</span> <span class="n">sSubparser</span> <span class="p">{</span>
        <span class="p">...</span>

        <span class="cm">/* public to the parser */</span>
        <span class="n">subparserRunDirection</span> <span class="n">direction</span><span class="p">;</span>

        <span class="kt">void</span> <span class="p">(</span><span class="o">*</span> <span class="n">inputStart</span><span class="p">)</span> <span class="p">(</span><span class="n">subparser</span> <span class="o">*</span><span class="n">s</span><span class="p">);</span>
        <span class="kt">void</span> <span class="p">(</span><span class="o">*</span> <span class="n">inputEnd</span><span class="p">)</span> <span class="p">(</span><span class="n">subparser</span> <span class="o">*</span><span class="n">s</span><span class="p">);</span>
        <span class="kt">void</span> <span class="p">(</span><span class="o">*</span> <span class="n">exclusiveSubparserChosenNotify</span><span class="p">)</span> <span class="p">(</span><span class="n">subparser</span> <span class="o">*</span><span class="n">s</span><span class="p">,</span> <span class="kt">void</span> <span class="o">*</span><span class="n">data</span><span class="p">);</span>
<span class="p">};</span>
</pre></div>
</div>
<p>A subparser must fill the fields of <cite>subparser</cite>.</p>
<p><cite>direction</cite> field specifies how the subparser is called.  If a
subparser runs exclusively and is chosen in top down way, set
<cite>SUBPARSER_SUB_RUNS_BASE</cite> flag. If a subparser runs coexisting way and
is chosen in bottom up way, set <cite>SUBPARSER_BASE_RUNS_SUB</cite>.  Use
<cite>SUBPARSER_BI_DIRECTION</cite> if The both cases can be considered.</p>
<p>SystemdUnit parser runs as a subparser of iniconf base parser.
SystemdUnit parser specifies <cite>SUBPARSER_SUB_RUNS_BASE</cite> because
unit files of systemd have very specific file extensions though
they are written in iniconf syntax. Therefore we expect SystemdUnit
parser is chosen in top down way. The same logic is applicable to
YumRepo parser.</p>
<p>Autoconf parser specifies <cite>SUBPARSER_BI_DIRECTION</cite>. For input
file having name <em>configure.ac</em>, by pattern matching, Autoconf parser
is chosen in top down way. In other hand, for file name <em>foo.m4</em>,
Autoconf parser can be chosen in bottom up way.</p>
<p><cite>inputStart</cite> is called before the base parser starting parsing a new input file.
<cite>inputEnd</cite> is called after the base parser finishing parsing the input file.
Universal-ctags main part calls these methods. Therefore, a base parser doesn’t
have to call them.</p>
<p><cite>exclusiveSubparserChosenNotify</cite> is called when a parser is chosen
as an exclusive parser. Calling this method is a job of a base parser.</p>
</div>
<div class="section" id="extending-subparser-type">
<h5>Extending <cite>subparser</cite> type<a class="headerlink" href="#extending-subparser-type" title="Permalink to this headline">¶</a></h5>
<p>The m4 parser extends <cite>subparser</cite> type like following:</p>
<p><em>parsers/m4.h</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">typedef</span> <span class="k">struct</span> <span class="n">sM4Subparser</span> <span class="n">m4Subparser</span><span class="p">;</span>
<span class="k">struct</span> <span class="n">sM4Subparser</span> <span class="p">{</span>
        <span class="n">subparser</span> <span class="n">subparser</span><span class="p">;</span>

        <span class="kt">bool</span> <span class="p">(</span><span class="o">*</span> <span class="n">probeLanguage</span><span class="p">)</span> <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">token</span><span class="p">);</span>

        <span class="cm">/* return value: Cork index */</span>
        <span class="kt">int</span>  <span class="p">(</span><span class="o">*</span> <span class="n">newMacroNotify</span><span class="p">)</span> <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">token</span><span class="p">);</span>

        <span class="kt">bool</span> <span class="p">(</span><span class="o">*</span> <span class="n">doesLineCommentStart</span><span class="p">)</span>   <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4</span><span class="p">,</span> <span class="kt">int</span> <span class="n">c</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="n">token</span><span class="p">);</span>
        <span class="kt">bool</span> <span class="p">(</span><span class="o">*</span> <span class="n">doesStringLiteralStart</span><span class="p">)</span> <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4</span><span class="p">,</span> <span class="kt">int</span> <span class="n">c</span><span class="p">);</span>
<span class="p">};</span>
</pre></div>
</div>
<p>Put <cite>subparser</cite> as the first member of the extended struct(here sM4Subparser).
In addition the first field, 4 methods are defined in the extended struct.</p>
<p>Till choosing a subparser for the current input file, the m4 parser calls
<cite>probeLanguage</cite> method of its subparsers each time when find a token
in the input file. A subparser returns <cite>true</cite> if it recognizes the
input file is for the itself by analyzing tokens passed from the
base parser.</p>
<p><em>parsers/autoconf.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">extern</span> <span class="n">parserDefinition</span><span class="o">*</span> <span class="nf">AutoconfParser</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
        <span class="k">static</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="k">const</span> <span class="n">patterns</span> <span class="p">[]</span> <span class="o">=</span> <span class="p">{</span> <span class="s">&quot;configure.in&quot;</span><span class="p">,</span> <span class="nb">NULL</span> <span class="p">};</span>
        <span class="k">static</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="k">const</span> <span class="n">extensions</span> <span class="p">[]</span> <span class="o">=</span> <span class="p">{</span> <span class="s">&quot;ac&quot;</span><span class="p">,</span> <span class="nb">NULL</span> <span class="p">};</span>
        <span class="n">parserDefinition</span><span class="o">*</span> <span class="k">const</span> <span class="n">def</span> <span class="o">=</span> <span class="n">parserNew</span><span class="p">(</span><span class="s">&quot;Autoconf&quot;</span><span class="p">);</span>

        <span class="k">static</span> <span class="n">m4Subparser</span> <span class="n">autoconfSubparser</span> <span class="o">=</span> <span class="p">{</span>
                <span class="p">.</span><span class="n">subparser</span> <span class="o">=</span> <span class="p">{</span>
                        <span class="p">.</span><span class="n">direction</span> <span class="o">=</span> <span class="n">SUBPARSER_BI_DIRECTION</span><span class="p">,</span>
                        <span class="p">.</span><span class="n">exclusiveSubparserChosenNotify</span> <span class="o">=</span> <span class="n">exclusiveSubparserChosenCallback</span><span class="p">,</span>
                <span class="p">},</span>
                <span class="p">.</span><span class="n">probeLanguage</span>  <span class="o">=</span> <span class="n">probeLanguage</span><span class="p">,</span>
                <span class="p">.</span><span class="n">newMacroNotify</span> <span class="o">=</span> <span class="n">newMacroCallback</span><span class="p">,</span>
                <span class="p">.</span><span class="n">doesLineCommentStart</span> <span class="o">=</span> <span class="n">doesLineCommentStart</span><span class="p">,</span>
                <span class="p">.</span><span class="n">doesStringLiteralStart</span> <span class="o">=</span> <span class="n">doesStringLiteralStart</span><span class="p">,</span>
        <span class="p">};</span>
</pre></div>
</div>
<p><cite>probeLanguage</cite> function defined in <em>autoconf.c</em> is connected to
the <cite>probeLanguage</cite> member of <cite>autoconfSubparser</cite>. The <cite>probeLanguage</cite> function
of Autoconf is very simple:</p>
<p><em>parsers/autoconf.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="kt">bool</span> <span class="nf">probeLanguage</span> <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">token</span><span class="p">)</span>
<span class="p">{</span>
        <span class="k">return</span> <span class="n">strncmp</span> <span class="p">(</span><span class="n">token</span><span class="p">,</span> <span class="s">&quot;m4_&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span>
                <span class="o">||</span> <span class="n">strncmp</span> <span class="p">(</span><span class="n">token</span><span class="p">,</span> <span class="s">&quot;AC_&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span>
                <span class="o">||</span> <span class="n">strncmp</span> <span class="p">(</span><span class="n">token</span><span class="p">,</span> <span class="s">&quot;AM_&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span>
                <span class="o">||</span> <span class="n">strncmp</span> <span class="p">(</span><span class="n">token</span><span class="p">,</span> <span class="s">&quot;AS_&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span>
                <span class="o">||</span> <span class="n">strncmp</span> <span class="p">(</span><span class="n">token</span><span class="p">,</span> <span class="s">&quot;AH_&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span>
                <span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>This function checks the prefix of passed tokens. If known
prefix is found, Autoconf assumes this is an Autoconf input
and returns <cite>true</cite>.</p>
<p><em>parsers/m4.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="p">(</span><span class="n">m4tmp</span><span class="o">-&gt;</span><span class="n">probeLanguage</span>
        <span class="o">&amp;&amp;</span> <span class="n">m4tmp</span><span class="o">-&gt;</span><span class="n">probeLanguage</span> <span class="p">(</span><span class="n">m4tmp</span><span class="p">,</span> <span class="n">token</span><span class="p">))</span>
<span class="p">{</span>
        <span class="n">chooseExclusiveSubparser</span> <span class="p">((</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="p">)</span><span class="n">tmp</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span>
        <span class="n">m4found</span> <span class="o">=</span> <span class="n">m4tmp</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The m4 parsers calls probeLanguage function of a subparser. If <cite>true</cite>
is returned <cite>chooseExclusiveSubparser</cite> function which is defined
in the main part. <cite>chooseExclusiveSubparser</cite> calls
<cite>exclusiveSubparserChosenNotify</cite> method of the chosen subparser.</p>
<p>The method is implemented in Autoconf subparser like following:</p>
<p><em>parsers/autoconf.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="kt">void</span> <span class="nf">exclusiveSubparserChosenCallback</span> <span class="p">(</span><span class="n">subparser</span> <span class="o">*</span><span class="n">s</span><span class="p">,</span> <span class="kt">void</span> <span class="o">*</span><span class="n">data</span><span class="p">)</span>
<span class="p">{</span>
        <span class="n">setM4Quotes</span> <span class="p">(</span><span class="sc">&#39;[&#39;</span><span class="p">,</span> <span class="sc">&#39;]&#39;</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
</div>
<p>It changes quote characters of the m4 parser.</p>
</div>
<div class="section" id="making-a-tag-in-a-subparser">
<h5>Making a tag in a subparser<a class="headerlink" href="#making-a-tag-in-a-subparser" title="Permalink to this headline">¶</a></h5>
<p>Via calling callback functions defined in subparsers, their base parser
gives chance to them making tag entries.</p>
<p>The m4 parser calls <cite>newMacroNotify</cite> method when it finds an m4 macro is used.
The Autoconf parser connects <cite>newMacroCallback</cite> function defined in <em>parser/autoconf.c</em>.</p>
<p><em>parsers/autoconf.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="kt">int</span> <span class="nf">newMacroCallback</span> <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">token</span><span class="p">)</span>
<span class="p">{</span>
        <span class="kt">int</span> <span class="n">keyword</span><span class="p">;</span>
        <span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="n">CORK_NIL</span><span class="p">;</span>

        <span class="n">keyword</span> <span class="o">=</span> <span class="n">lookupKeyword</span> <span class="p">(</span><span class="n">token</span><span class="p">,</span> <span class="n">getInputLanguage</span> <span class="p">());</span>

        <span class="cm">/* TODO:</span>
<span class="cm">           AH_VERBATIM</span>
<span class="cm">         */</span>
        <span class="k">switch</span> <span class="p">(</span><span class="n">keyword</span><span class="p">)</span>
        <span class="p">{</span>
        <span class="k">case</span> <span class="nl">KEYWORD_NONE</span><span class="p">:</span>
                <span class="k">break</span><span class="p">;</span>
        <span class="k">case</span> <span class="nl">KEYWORD_init</span><span class="p">:</span>
                <span class="n">index</span> <span class="o">=</span> <span class="n">makeAutoconfTag</span> <span class="p">(</span><span class="n">PACKAGE_KIND</span><span class="p">);</span>
                <span class="k">break</span><span class="p">;</span>

<span class="p">...</span>

<span class="k">extern</span> <span class="n">parserDefinition</span><span class="o">*</span> <span class="n">AutoconfParser</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
        <span class="p">...</span>
        <span class="k">static</span> <span class="n">m4Subparser</span> <span class="n">autoconfSubparser</span> <span class="o">=</span> <span class="p">{</span>
                <span class="p">.</span><span class="n">subparser</span> <span class="o">=</span> <span class="p">{</span>
                        <span class="p">.</span><span class="n">direction</span> <span class="o">=</span> <span class="n">SUBPARSER_BI_DIRECTION</span><span class="p">,</span>
                        <span class="p">.</span><span class="n">exclusiveSubparserChosenNotify</span> <span class="o">=</span> <span class="n">exclusiveSubparserChosenCallback</span><span class="p">,</span>
                <span class="p">},</span>
                <span class="p">.</span><span class="n">probeLanguage</span>  <span class="o">=</span> <span class="n">probeLanguage</span><span class="p">,</span>
                <span class="p">.</span><span class="n">newMacroNotify</span> <span class="o">=</span> <span class="n">newMacroCallback</span><span class="p">,</span>
</pre></div>
</div>
<p>In <cite>newMacroCallback</cite> function, the Autoconf parser receives the name of macro
found by the base parser and analysis weather the macro is interesting
in the context of Autoconf language or not. If it is interesting name,
the Autoconf parser makes a tag for it.</p>
</div>
<div class="section" id="calling-methods-of-subparsers-from-a-base-parser">
<h5>Calling methods of subparsers from a base parser<a class="headerlink" href="#calling-methods-of-subparsers-from-a-base-parser" title="Permalink to this headline">¶</a></h5>
<p>A base parser can use <cite>foreachSubparser</cite> macro for accessing its
subparsers. A base should call <cite>enterSubparser</cite> before calling a
method of a subparser, and call <cite>leaveSubparser</cite> after calling the
method. The macro and functions are declare in <em>main/subparser.h</em> .</p>
<p><em>parsers/m4.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="n">m4Subparser</span> <span class="o">*</span> <span class="nf">maySwitchLanguage</span> <span class="p">(</span><span class="k">const</span> <span class="kt">char</span><span class="o">*</span> <span class="n">token</span><span class="p">)</span>
<span class="p">{</span>
        <span class="n">subparser</span> <span class="o">*</span><span class="n">tmp</span><span class="p">;</span>
        <span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4found</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>

        <span class="n">foreachSubparser</span> <span class="p">(</span><span class="n">tmp</span><span class="p">,</span> <span class="nb">false</span><span class="p">)</span>
        <span class="p">{</span>
                <span class="n">m4Subparser</span> <span class="o">*</span><span class="n">m4tmp</span> <span class="o">=</span> <span class="p">(</span><span class="n">m4Subparser</span> <span class="o">*</span><span class="p">)</span><span class="n">tmp</span><span class="p">;</span>

                <span class="n">enterSubparser</span><span class="p">(</span><span class="n">tmp</span><span class="p">);</span>
                <span class="k">if</span> <span class="p">(</span><span class="n">m4tmp</span><span class="o">-&gt;</span><span class="n">probeLanguage</span>
                        <span class="o">&amp;&amp;</span> <span class="n">m4tmp</span><span class="o">-&gt;</span><span class="n">probeLanguage</span> <span class="p">(</span><span class="n">m4tmp</span><span class="p">,</span> <span class="n">token</span><span class="p">))</span>
                <span class="p">{</span>
                        <span class="n">chooseExclusiveSubparser</span> <span class="p">(</span><span class="n">tmp</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span>
                        <span class="n">m4found</span> <span class="o">=</span> <span class="n">m4tmp</span><span class="p">;</span>
                <span class="p">}</span>
                <span class="n">leaveSubparser</span><span class="p">();</span>

                <span class="k">if</span> <span class="p">(</span><span class="n">m4found</span><span class="p">)</span>
                        <span class="k">break</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="k">return</span> <span class="n">m4found</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p><cite>foreachSubparser</cite> takes a variable having type <cite>subparser</cite>.
For each iteration, the value for the variable is updated.</p>
<p><cite>enterSubparser</cite> takes a variable having type <cite>subparser</cite>.  With the
calling <cite>enterSubparser</cite>, the current language(the value returned from
<cite>getInputLanguage</cite>) can be temporary switched to the language specified
with the variable. One of the effect of switching is that <cite>language</cite>
field of tags made in the callback function called between
<cite>enterSubparser</cite> and <cite>leaveSubparser</cite> is adjusted.</p>
</div>
<div class="section" id="registering-a-subparser-to-its-base-parser">
<h5>Registering a subparser to its base parser<a class="headerlink" href="#registering-a-subparser-to-its-base-parser" title="Permalink to this headline">¶</a></h5>
<p>Use <cite>DEPTYPE_SUBPARSER</cite> dependency in a subparser for registration.</p>
<p><em>parsers/autoconf.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">extern</span> <span class="n">parserDefinition</span><span class="o">*</span> <span class="nf">AutoconfParser</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
        <span class="n">parserDefinition</span><span class="o">*</span> <span class="k">const</span> <span class="n">def</span> <span class="o">=</span> <span class="n">parserNew</span><span class="p">(</span><span class="s">&quot;Autoconf&quot;</span><span class="p">);</span>

        <span class="k">static</span> <span class="n">m4Subparser</span> <span class="n">autoconfSubparser</span> <span class="o">=</span> <span class="p">{</span>
                <span class="p">.</span><span class="n">subparser</span> <span class="o">=</span> <span class="p">{</span>
                        <span class="p">.</span><span class="n">direction</span> <span class="o">=</span> <span class="n">SUBPARSER_BI_DIRECTION</span><span class="p">,</span>
                        <span class="p">.</span><span class="n">exclusiveSubparserChosenNotify</span> <span class="o">=</span> <span class="n">exclusiveSubparserChosenCallback</span><span class="p">,</span>
                <span class="p">},</span>
                <span class="p">.</span><span class="n">probeLanguage</span>  <span class="o">=</span> <span class="n">probeLanguage</span><span class="p">,</span>
                <span class="p">.</span><span class="n">newMacroNotify</span> <span class="o">=</span> <span class="n">newMacroCallback</span><span class="p">,</span>
                <span class="p">.</span><span class="n">doesLineCommentStart</span> <span class="o">=</span> <span class="n">doesLineCommentStart</span><span class="p">,</span>
                <span class="p">.</span><span class="n">doesStringLiteralStart</span> <span class="o">=</span> <span class="n">doesStringLiteralStart</span><span class="p">,</span>
        <span class="p">};</span>
        <span class="k">static</span> <span class="n">parserDependency</span> <span class="n">dependencies</span> <span class="p">[]</span> <span class="o">=</span> <span class="p">{</span>
                <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> <span class="n">DEPTYPE_SUBPARSER</span><span class="p">,</span> <span class="s">&quot;M4&quot;</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">autoconfSubparser</span> <span class="p">},</span>
        <span class="p">};</span>

        <span class="n">def</span><span class="o">-&gt;</span><span class="n">dependencies</span> <span class="o">=</span> <span class="n">dependencies</span><span class="p">;</span>
        <span class="n">def</span><span class="o">-&gt;</span><span class="n">dependencyCount</span> <span class="o">=</span> <span class="n">ARRAY_SIZE</span> <span class="p">(</span><span class="n">dependencies</span><span class="p">);</span>
</pre></div>
</div>
<p><cite>DEPTYPE_SUBPARSER</cite> is specified in the 0th element of`dependencies`
function static variable. In the next a literal string “M4” is
specified and <cite>autoconfSubparser</cite> follows. The intent of the code is
registering <cite>autoconfSubparser</cite> subparser definition to a base parser
named “M4”.</p>
<p><cite>dependencies</cite> function static variable must be assigned to
<cite>dependencies</cite> fields of a variable of <cite>parserDefinition</cite>.
The main part of Universal-ctags refers the field when
initializing parsers.</p>
<p><cite>[0]</cite> emphasizes this is “the 0th element”. The subparser may refer
the index of the array when the subparser calls
<cite>scheduleRunningBaseparser</cite>.</p>
</div>
<div class="section" id="scheduling-running-the-base-parser">
<h5>Scheduling running the base parser<a class="headerlink" href="#scheduling-running-the-base-parser" title="Permalink to this headline">¶</a></h5>
<p>For the case that a subparser is chosen in top down, the subparser
must call <cite>scheduleRunningBaseparser</cite> in the main <cite>parser</cite> method.</p>
<p><em>parsers/autoconf.c</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="kt">void</span> <span class="nf">findAutoconfTags</span><span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
        <span class="n">scheduleRunningBaseparser</span> <span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="p">}</span>

<span class="k">extern</span> <span class="n">parserDefinition</span><span class="o">*</span> <span class="nf">AutoconfParser</span> <span class="p">(</span><span class="kt">void</span><span class="p">)</span>
<span class="p">{</span>
        <span class="p">...</span>
        <span class="n">parserDefinition</span><span class="o">*</span> <span class="k">const</span> <span class="n">def</span> <span class="o">=</span> <span class="n">parserNew</span><span class="p">(</span><span class="s">&quot;Autoconf&quot;</span><span class="p">);</span>
        <span class="p">...</span>
        <span class="k">static</span> <span class="n">parserDependency</span> <span class="n">dependencies</span> <span class="p">[]</span> <span class="o">=</span> <span class="p">{</span>
                <span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> <span class="n">DEPTYPE_SUBPARSER</span><span class="p">,</span> <span class="s">&quot;M4&quot;</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">autoconfSubparser</span> <span class="p">},</span>
        <span class="p">};</span>

        <span class="n">def</span><span class="o">-&gt;</span><span class="n">dependencies</span> <span class="o">=</span> <span class="n">dependencies</span><span class="p">;</span>
        <span class="p">...</span>
        <span class="n">def</span><span class="o">-&gt;</span><span class="n">parser</span> <span class="o">=</span> <span class="n">findAutoconfTags</span><span class="p">;</span>
        <span class="p">...</span>
        <span class="k">return</span> <span class="n">def</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>A subparser can do nothing actively. A base parser makes its subparser
work b calling methods of the subparser.  Therefor a subparser must
run its base parser when the subparser is chosen in a top down way,
The main part prepares <cite>scheduleRunningBaseparser</cite> function for the purpose.</p>
<p>A subparser should call the function from <cite>parser</cite> method of <cite>parserDefinition</cite>
of the subparser. <cite>scheduleRunningBaseparser</cite> takes an integer. It specifies
an index of the dependency which is used for registering the subparser.</p>
</div>
<div class="section" id="command-line-interface">
<h5>Command line interface<a class="headerlink" href="#command-line-interface" title="Permalink to this headline">¶</a></h5>
<p>Running subparser can be controlled with <cite>s</cite> extras flag.
By default it is enabled. To turning off the feature running
subparser, specify <cite>–extras=-s</cite>.</p>
<p>When <cite>–extras=+E</cite> option given, a tag entry recorded by a subparser
is marked as follows:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span>TMPDIR  input.ac        /^AH_TEMPLATE([TMPDIR],$/;&quot;     template        extras:subparser        end:4
</pre></div>
</div>
<p>See also <a class="reference internal" href="news.html#defining-subparsers"><span class="std std-ref">Defining a subparser</span></a>.</p>
</div>
</div>
</div>
<div class="section" id="examples-of-sub-base-combinations">
<h3>Examples of sub/base combinations<a class="headerlink" href="#examples-of-sub-base-combinations" title="Permalink to this headline">¶</a></h3>
<div class="section" id="automake-make-parser-combination">
<h4>Automake/Make parser combination<a class="headerlink" href="#automake-make-parser-combination" title="Permalink to this headline">¶</a></h4>
<p>Simply to say the syntax of Automake is the subset of Make.  However,
the Automake parser has interests in Make macros having special
suffixes: “_PROGRAMS”, “_LTLIBRARIES”, and “_SCRIPTS” so on.</p>
<p>Here is an example of input for Automake:</p>
<div class="highlight-Make"><div class="highlight"><pre><span></span><span class="nv">bin_PROGRAMS</span> <span class="o">=</span> ctags
<span class="nv">ctags_CPPFLAGS</span> <span class="o">=</span>    <span class="se">\</span>
        -I.         <span class="se">\</span>
        -I<span class="k">$(</span>srcdir<span class="k">)</span> <span class="se">\</span>
        -I<span class="k">$(</span>srcdir<span class="k">)</span>/main
</pre></div>
</div>
<p>From the point of the view of the Make parser, bin_PROGRAMS is a just
a macro; the Make parser tags bin_PROGRAMS as a macro. The Make parser
doesn’t tag “ctags” being right side of <cite>=</cite> because it is not a new
name: just a value assigned to bin_PROGRAMS. However, for the Automake
parser “ctags” is a new name; the Automake parser tags “ctags” with
kind “Program”. The Automake parser can tag it with getting help from
the Make parser.</p>
<p>The Automake parser is an exclusive subparser. It is chosen in top
down way; an input file name “Makefile.am” gives enough information for
choosing the Automake parser.</p>
<p>To give chances to the Automake parser to capture Automake own
definitions, The Make parser provides following interface in
<em>parsers/make.h</em>:</p>
<div class="highlight-C"><div class="highlight"><pre><span></span><span class="k">struct</span> <span class="n">sMakeSubparser</span> <span class="p">{</span>
        <span class="n">subparser</span> <span class="n">subparser</span><span class="p">;</span>

        <span class="kt">void</span> <span class="p">(</span><span class="o">*</span> <span class="n">valueNotify</span><span class="p">)</span> <span class="p">(</span><span class="n">makeSubparser</span> <span class="o">*</span><span class="n">s</span><span class="p">,</span> <span class="kt">char</span><span class="o">*</span> <span class="n">name</span><span class="p">);</span>
        <span class="kt">void</span> <span class="p">(</span><span class="o">*</span> <span class="n">directiveNotify</span><span class="p">)</span> <span class="p">(</span><span class="n">makeSubparser</span> <span class="o">*</span><span class="n">s</span><span class="p">,</span> <span class="kt">char</span><span class="o">*</span> <span class="n">name</span><span class="p">);</span>
        <span class="kt">void</span> <span class="p">(</span><span class="o">*</span> <span class="n">newMacroNotify</span><span class="p">)</span> <span class="p">(</span><span class="n">makeSubparser</span> <span class="o">*</span><span class="n">s</span><span class="p">,</span>
                                 <span class="kt">char</span><span class="o">*</span> <span class="n">name</span><span class="p">,</span>
                                 <span class="kt">bool</span> <span class="n">withDefineDirective</span><span class="p">,</span>
                                 <span class="kt">bool</span> <span class="n">appending</span><span class="p">);</span>
<span class="p">};</span>
</pre></div>
</div>
<p>The Automake parser defines methods for tagging Automake own definitions
in a <cite>struct sMakeSubparser</cite> type variable, and runs the Make parser by
calling <cite>scheduleRunningBaseparser</cite> function.</p>
<p>The Make parser tags Make own definitions in an input file.  In
addition Make parser calls the methods during parsing the input file.</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">$</span> ./ctags --fields<span class="o">=</span>+lK  --extras<span class="o">=</span>+r -o - Makefile.am
<span class="go">bin  Makefile.am     /^bin_PROGRAMS = ctags$/;&quot;      directory       language:Automake</span>
<span class="go">bin_PROGRAMS Makefile.am     /^bin_PROGRAMS = ctags$/;&quot;      macro   language:Make</span>
<span class="go">ctags        Makefile.am     /^bin_PROGRAMS = ctags$/;&quot;      program language:Automake       directory:bin</span>
<span class="go">ctags_CPPFLAGS       Makefile.am     /^ctags_CPPFLAGS =    \\$/;&quot;    macro   language:Make</span>
</pre></div>
</div>
<p><cite>bin_PROGRAMS</cite> and <cite>ctags_CPPFLAGS</cite> are tagged as macros of Make.
In addition <cite>bin</cite> is tagged as directory, and <cite>ctags</cite> as program of Automake.</p>
<p><cite>bin</cite> is tagged in a callback function assigned to <cite>newMacroFound</cite> method.
<cite>ctags</cite> is tagged in a callback function assigned to <cite>valuesFound</cite> method.</p>
<p><cite>–extras=+r</cite> is used in the example. <cite>r</cite> extra is needed to
tag <cite>bin</cite>. <cite>bin</cite> is not defined in the line, <cite>bin_PROGRAMS =</cite>.
<cite>bin</cite> is referenced as a name of directory where programs are
stored. Therefore <cite>r</cite> is needed.</p>
<p>For tagging <cite>ctags</cite>, the Automake parser must recognize
<cite>bin</cite> in <cite>bin_PROGRAMS</cite> first. <cite>ctags</cite> is tagged
because it is specified as a value for <cite>bin_PROGRAMS</cite>.
As the result <cite>r</cite> is also needed to tag <cite>ctags</cite>.</p>
<p>Only Automake related tags are emitted if Make parser is
disabled.</p>
<div class="highlight-console"><div class="highlight"><pre><span></span><span class="gp">$</span> ./ctags --languages<span class="o">=</span>-Make --fields<span class="o">=</span>+lKr --extras<span class="o">=</span>+r -o - Makefile.am
<span class="go">bin     Makefile.am     /^bin_PROGRAMS = ctags$/;&quot;      directory       language:Automake       role:program</span>
<span class="go">ctags   Makefile.am     /^bin_PROGRAMS = ctags$/;&quot;      program language:Automake       directory:bin</span>
</pre></div>
</div>
</div>
<div class="section" id="autoconf-m4-parser-combination">
<h4>Autoconf/M4 parser combination<a class="headerlink" href="#autoconf-m4-parser-combination" title="Permalink to this headline">¶</a></h4>
<p>Universal-ctags uses m4 parser as a base parser and Autoconf parse as
a sub parser for <cite>configure.ac</cite> input file.</p>
<div class="highlight-Autoconf"><div class="highlight"><pre><span></span>AC_DEFUN([PRETTY_VAR_EXPAND],
          [$(eval &quot;$as_echo_n&quot; $(eval &quot;$as_echo_n&quot; &quot;${$1}&quot;))])
</pre></div>
</div>
<p>The m4 parser finds no definition here.  However, Autoconf parser finds
<cite>PRETTY_VAR_EXPAND</cite> as a macro definition. Syntax like <cite>(…)</cite> is part
of M4 language. So Autoconf parser is implemented as a sub parser of
m4 parser. The most parts of tokens in input files are handled by
M4. Autoconf parser gives hints for parsing <cite>configure.ac</cite> and
registers callback functions to
Autoconf parser.</p>
</div>
</div>
</div>
</div>


          </div>
        </div>
      </div>
      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
        <div class="sphinxsidebarwrapper">
  <h3><a href="index.html">Table Of Contents</a></h3>
  <ul>
<li><a class="reference internal" href="#">Running multiple parsers on an input file</a><ul>
<li><a class="reference internal" href="#applying-a-parser-to-specified-areas-of-input-file-guest-host">Applying a parser to specified areas of input file (guest/host)</a><ul>
<li><a class="reference internal" href="#examples-of-guest-host-combinations">Examples of guest/host combinations</a><ul>
<li><a class="reference internal" href="#css-javascript-html-parser-combination">{CSS,JavaScript}/HTML parser combination</a></li>
<li><a class="reference internal" href="#c-yacc-parser-combination">C/Yacc parser combination</a></li>
<li><a class="reference internal" href="#pod-perl-parser-combination">Pod/Perl parser combination</a></li>
</ul>
</li>
<li><a class="reference internal" href="#api-for-running-a-parser-in-an-area">API for running a parser in an area</a></li>
</ul>
</li>
<li><a class="reference internal" href="#tagging-definitions-of-higher-upper-level-language-sub-base">Tagging definitions of higher(upper) level language (sub/base)</a><ul>
<li><a class="reference internal" href="#background">Background</a></li>
<li><a class="reference internal" href="#terms">Terms</a><ul>
<li><a class="reference internal" href="#base-parser-and-subparser">Base parser and subparser</a></li>
<li><a class="reference internal" href="#top-down-parser-choice-and-bottom-up-parser-choice">Top down parser choice and bottom up parser choice</a></li>
<li><a class="reference internal" href="#exclusive-subparser-and-coexisting-subparser">Exclusive subparser and coexisting subparser</a></li>
</ul>
</li>
<li><a class="reference internal" href="#api-for-making-a-combination-of-base-parser-and-subparsers">API for making a combination of base parser and subparsers</a><ul>
<li><a class="reference internal" href="#outline">Outline</a></li>
<li><a class="reference internal" href="#details">Details</a><ul>
<li><a class="reference internal" href="#fields-of-subparser-type">Fields of <cite>subparser</cite> type</a></li>
<li><a class="reference internal" href="#extending-subparser-type">Extending <cite>subparser</cite> type</a></li>
<li><a class="reference internal" href="#making-a-tag-in-a-subparser">Making a tag in a subparser</a></li>
<li><a class="reference internal" href="#calling-methods-of-subparsers-from-a-base-parser">Calling methods of subparsers from a base parser</a></li>
<li><a class="reference internal" href="#registering-a-subparser-to-its-base-parser">Registering a subparser to its base parser</a></li>
<li><a class="reference internal" href="#scheduling-running-the-base-parser">Scheduling running the base parser</a></li>
<li><a class="reference internal" href="#command-line-interface">Command line interface</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#examples-of-sub-base-combinations">Examples of sub/base combinations</a><ul>
<li><a class="reference internal" href="#automake-make-parser-combination">Automake/Make parser combination</a></li>
<li><a class="reference internal" href="#autoconf-m4-parser-combination">Autoconf/M4 parser combination</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>

  <h4>Previous topic</h4>
  <p class="topless"><a href="guessing.html"
                        title="previous chapter">Choosing a proper parser in ctags</a></p>
  <h4>Next topic</h4>
  <p class="topless"><a href="building.html"
                        title="next chapter">Building ctags</a></p>
<div id="searchbox" style="display: none" role="search">
  <h3>Quick search</h3>
    <form class="search" action="search.html" method="get">
      <div><input type="text" name="q" /></div>
      <div><input type="submit" value="Go" /></div>
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="building.html" title="Building ctags"
             >next</a> |</li>
        <li class="right" >
          <a href="guessing.html" title="Choosing a proper parser in ctags"
             >previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="index.html">Universal Ctags 0.3.0 documentation</a> &#187;</li> 
      </ul>
    </div>
    <div class="footer" role="contentinfo">
        &#169; Copyright 2015, Universal Ctags Team.
      Last updated on 19 Dec 2017.
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.4.
    </div>
  </body>
</html>